IN THE SPECIFICATION : 

Please replace paragraph numbers [0001] , [0003]- 
[0006], [0008]-[0012] , [0014], [0029], [ 0031 ]-[ 0034 ] , [0039], 
[0043], [0045], [0046], [ 0049 ]-[ 0053 ] , [0056], [0057], 
[0060], [0062], [0066], [0070], and [ 0072 ]-[ 0074 ] with the 
corresponding paragraph below. A marked-up version showing 
the amendments is included in the attached Appendix. 

[0001 ] The present invention relates to an image processing apparatus suitable 
for preventing counterfeiting of copy-prohibited objects, such as paper money, 
securities and the like. 



[0003] A copier is basically designed to faithfully copy an object mounted on an 
original-mount, except for a copy-prohibited object, such as paper money or the 
like. Accordingly, if an image which resembles paper money (an image other 
than paper money which can be legally copied) is input, the copier must 
recognize that the image is not paper money, and perform a copying operation of 
the original image. Hence, very precise recognition accuracy is requested for 
determining whether or not an image to be recognized is an image for which 
counterfeiting is to be prevented. 

[0004] Recently, the quality of an image obtained by a color scanner or a color 
printer, which is less expensive than a color copier, is greatly improved, and it 



becomes possible to perform a counterfeiting operation by connecting such an 
apparatus to a personal computer, i.e., by combining apparatuses other than a 
color copier. Accordingly, even in an image processing system using an 
inexpensive color scanner or color printer, countermeasures for preventing 
counterfeiting of paper money, securities and the like by mounting an image 
recognition function or the like have been desired. 

[0005] Various methods for determining whether or not an original is a copy- 
prohibited object have been proposed. In one method, a color spectrum 
distribution of image data obtained by scanning an original is compared with 
data registered in advance in a ROM (read-only memory), based on each pixel 
value (a set of R, G and B values corresponding to three color signals) in the 
image data (for example, by obtaining a sum of the absolute values of difference 
values between respective pixel values and stored data, or calculating a cross- 
correlation value between distributions). In another method, an evaluation value 
is calculated by comparing a synthetic image pattern of a part or the entirety of 
an original with pattern data registered in advance in a ROM or the like. 
[0006] Recently, it became possible to incorporate information indicating a 
copy-prohibited object within an original, serving as a printed matter, using a 
technique called "digital watermark That is, when printing an original, serving as 
a copy-prohibited object, a printed matter is obtained from image data 
(electronic image information) including copy-prohibited-object information 
generated by embedding second digital information (sub-information) indicating 




a copy-prohibited object in image data (electronic image information), serving as 
original first digital information (main information). This copy-prohibited-object 
information (sub-information) can be extracted from the image data including 
the copy-prohibited-object information before printing the original. Furthermore, 
if consistency between conditions when forming the printed matter from the 
image data (electronic image information) including the copy-prohibited-object 
information and conditions in reading and extraction processing is obtained, it is 
also possible to extract the copy-prohibited-object information from electronic 
image data obtained by reading the printed matter (original) formed from 
electronic image information including the copy-prohibited-object information by 
an original-reading device, such as a color scanner or the like. In consideration of 
these features, application of the technique called "digital watermark" to prevent 
illegal copying of an original, serving as a printed matter, has been attracting 
notice. 



[0008] Accordingly, when reading a printed matter (an original), formed from 
electronic image information having embedded digital-watermark information, 
by an original-reading device, such as a color scanner or the like, the direction 
the original is mounted on the surface of an original-mount greatly influences the 
difficulty in extraction of digital-watermark information. 
[0009] When recognizing a copy-prohibited object by comparing a color 
spectrum distribution of image data obtained by scanning an original with data 
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registered in advance in a ROM, or by calculating an evaluation value by 
comparing a synthetic image pattern of a part or the entirety of the original with 
pattern data registered in advance in a ROM or the like, if the direction (angle) or 
the position of the mounted original is not fixed, the total amount of calculation 
operations for comparison with spectrum data registered in advance or image- 
pattern data tends to greatly increase in order to deal with various directions and 
positions. 

[0010] In an inexpensive color scanner or color printer, it is desirable to prevent 
counterfeiting of a copy-prohibited object with a cost lower than in the case of 
using a relatively expensive color copier. For that purpose, it is preferable to 
recognize a copy-prohibited object by mainly using software instead of mainly 
using hardware having a large amount of electronic circuits and the like. 
[001 1 ] On the other hand, when determining a copy-prohibited image requiring 
precise recognition accuracy mainly by software, the total amount of calculation 
operations is large, and the processing time required for the recognition and 
determination tends to be greatly increased. 

[0012] That is, if the amount of calculation operations for the recognition and 
determination is reduced, it generally tends to be difficult to determine a copy- 
prohibited image requiring precise recognition accuracy such that, for example, 
it becomes difficult to detect a copy-prohibited object, such as paper money or 
the like, read by intentionally changing reading conditions (such as a mounting 
angle or a mounting position of an original on a reading surface), or that when an 
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image resembling paper money (an image other than paper money which is 
legally allowed to be copied) is input, the image is erroneously recognized as a 
copy-prohibited image and normal image output is not performed, thereby 
impairing the original function of an image processing system. 



[001 4] It is another object of the present invention to provide an image 
processing apparatus and a method for controlling the same which allow 
recognition of whether or not an image is a copy-prohibited object that is more 
precise than in the prior art, even in recognition/determination processing having 
a relatively small amount of calculation operations in a personal computer 
system using an inexpensive image scanner or color printer. 



[0029] FIG. 1 1 is a flowchart illustrating another operation procedure in the 
seventh embodiment. 



[0031 ] An image processing apparatus proposed in the following embodiments 
of the present invention includes, as shown in FIG. 1, an image-data acquisition 
unit 120 for inputting image data from an image-data source 110, such as a color 
scanner or the like, a copy-prohibited-object recognition unit 130 for determining 
whether or not the image data obtained by the image-data acquisition unit 120 is 
image data obtained by reading a copy-prohibited object, a retry determination 
unit 160 including a difficulty calculation unit 140 for calculating the difficulty in 
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determining whether or not the image data represents a copy-prohibited object, 
and a difficulty determination unit 150 for determining the difficulty from data 
obtained from the difficulty calculation unit 140, a retry-information transmission 
unit 1 70 for transmitting retry information, and the like. A retry-information 
reception unit 180, such as a display device or the like, receives the retry 
information, and displays, for example, a request (or recommendation) to again 
read an original by a scanner or the like to an operator or the like of an image 
processing system. A control unit 200 controls the operations of the entire image 
processing system. 

[0032] In the above-described configuration, each functional block operates in 
the following manner. By the operator operation on an instruction unit (not 
shown) of the image processing system, image data obtained by performing 
photoelectric scanning of an original set on the image-data source 1 10 is input to 
the copy-prohibited-object recognition unit 130 and the difficulty calculation unit 
140 via the image-data acquisition unit 120. The difficulty calculation unit 140 
calculates the difficulty in determining whether or not the input image data (may 
also be attribute data relating to the data, or the like) represents a copy- 
prohibited object. The calculated data of difficulty is output to the difficulty 
determination unit 150. The difficulty determination unit 150 determines the 
difficulty from data obtained from the difficulty calculation unit 140, and outputs 
the result of the determination to the control unit 200. When the result of the 
determination indicates that it is difficult to determine whether or not the image 
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data represents a copy-prohibited object and it is preferable to again read the 
original by a device for reading (generating) image data, such as a scanner or the 
like, the control unit 200 outputs a signal to request (or recommend) to again 
read the original by the image-data reading (generating) device to the retry- 
information reception unit 180 via the retry-information transmission unit 1 70 , in 
order to obtain an input image for which it is easier to determine a copy- 
/\y prohibited object. 

[0033] In the following first embodiment of the present invention, a description 
will be provided of a case in which information indicating a copy-prohibited 
object included within an original is formed according to a technique called 
"digital watermark " That is, it is assumed that an original, serving as a copy- 
prohibited object, is formed from image data (electronic image information) 
including copy-prohibited-object information generated by embedding second 
digital information indicating a copy-prohibited object in image data (electronic 
information), serving as first digital information to be formed as a printed matter. 
[0034] The digital watermark may have any appropriate form, such as an 
invisible digital watermark in which information is embedded in spatial 
frequencies of image data, a visible digital watermark in which information is 
embedded according to an arrangement of yellow (or yellow-type) dots which 
are difficult to be seen by human eyes, or the like. 
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[0039] FIG. 3 is a block diagram illustrating a principal portion in the 
configuration shown in FIG. 2 as functional modules. In FIG. 3, there are shown a 
CPU (central processing unit) 1 1, a RAM (random access memory) 12, a ROM 13, 
a display control unit 14, a display 15, an operation input device 16 including a 
keyboard, a mouse and the like, an input/output control device (hereinafter 
abbreviated as an 10 17 for connection to the operation input unit 16, an 
external storage device 18, such as a hard-disk device or the like, an I/O 19 for 
connection to the external storage device 18, a bus 20, a color image scanner 21, 
an I/O 22 for connection to an image input device, such as a color image scanner 
or the like, through the connection cable 3, and an interface unit 23 operating as 
an interface with communication means, such as a network or the like. 



[0043] In step S30, it is determined whether or not the read image has an angle 
(at least a predetermined angle0 th ) at which detection of a copy-prohibited 
image is difficult in recognition/determination processing with a relatively small 
amount of calculation operations. If the result of the determination in step S30 is 
affirmative, the process proceeds to step S40. If the result of the determination is 
negative, the process proceeds to step S50. 




[0045] In step S60, the presence of characteristics of a copy-prohibited object in 
the image data obtained in step S50 is checked. The process then proceeds to 
step S70. 
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[0046] In step S70, it is determined whether or not the image is a copy- 
prohibited image based on the result of the check in step S60. If the result of the 
determination in step S70 is affirmative, the process proceeds to step S80. If the 
result of the determination in step S70 is negative, the process proceeds to step 
S90. 



[0049] In step S40, the CPU 1 1 outputs a signal indicating that the mounting 
angle 0 of the original on the reading surface of the scanner is an angle (at least 
0 th ) at which detection of a copy-prohibited image is difficult, and requesting to 
again read the original by changing the mounting state, to the display device 15 
via the display control unit 14. Upon completion of step S40, the series of 
processing is terminated. In accordance with the result of the output, the display 
device 15 displays the mounting angle of the original in the form of an image. 
[0050] In the above-described step S80, the operation that (4) image data is not 
output at all may, of course, be performed instead of outputting a certain image 
data file. The operation that (5) a signal indicating a warning that the original is a 
copy-prohibited object is output to the display device 15 via the display control 
unit 14 may, of course, also be performed. 

[0051 ] In the processing of calculating the mounting angle of the original on the 
original-reading surface performed in step S20, the angle of the original is 
detected by detecting the positions of edges of the original for each scanning line 
by detecting the difference between the pixel density value when reading the 

10 



platen cover of the scanner and the pixel density value when reading the 
background portion of the original, and detecting the positions of the four corners 
of the original based on information relating to changes in the positions of the 
edges of the original. 

[0052] The check of the characteristics of a copy-prohibited object in step S60 
is performed by extracting the characteristics from image data obtained by 
performing photoelectric scanning of the original, serving as a printed matter 
including information relating to a copy-prohibited object using the above- 
described digital-watermark technique, by the scanner. 
[0053] In step S70, it is determined whether or not information relating to a 
copy-prohibited object has been extracted in step S60. If the result of the 
determination in step S70 is negative, the process proceeds to step S90. If the 
result of the determination in step S70 is affirmative, the process proceeds to step 
S80. 



[0056] Although in the above-described first embodiment, the check of the 
characteristics of a copy-prohibited object in step S60 is performed by extracting 
the characteristics from image data obtained by performing photoelectric 
scanning of the original, serving as a printed matter including copy-prohibited- 
object information using the digital-watermark technique, any other appropriate 
approach may also be adopted. The check of the characteristics of a copy- 
prohibited object in step S60 may be performed by comparing a color-spectrum 



distribution of image data obtained by scanning the original with data registered 
in advance in the ROM 13, based on each pixel value (a set of R, G and B values 
corresponding to three-color signals) in the image data, or comparing the 
synthetic image pattern of a part or the entirety of the original with pattern data 
registered in advance in a ROM or the like. In this case, in step S70, the 
evaluation value (the sum of the absolute values of difference values, the cross- 
correlation value between distributions, or the like) is determined based on a 
predetermined threshold (for example, if the sum of the absolute values of 
difference values does not exceed the threshold, the image is determined to be a 
copy-prohibited object, and if the sum exceeds the threshold, the image is 
determined not to be a copy-prohibited object, or if the cross-correlation value 
exceeds the threshold, the image is determined to be a copy-prohibited object, 
and if the cross-correlation value does not exceed the threshold, the image is 
determined not to be a copy-prohibited object). 

[0057] In the above-described first and second embodiments, the calculation of 
the mounting angle 0 of the original (calculation of the difficulty in extraction of 
copy-prohibited-object information) is performed in steps S20 and S30, and a 
determination of whether or not extraction of copy-prohibited-object information 
is difficult (determination of difficulty) is performed from the obtained angle. 
However, any other appropriate approach may be adopted. The process of 
obtaining the mounting angle of the original in step S20 may be replaced by 
processing of obtaining the positions of the four comers of the original in the 
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digital image obtained by reading the original on the original-mount (platen) and 
calculating the mounting position of the original (the coordinates of the four 
comers of the original) as shown in step S21 of FIG. 5, and the process of step 
S30 in FIG. 4 may be replaced by a determination of whether or not the mounting 
position of the original coincides with a predetermined standard position as 
shown in step S31 of FIG. 5. If the result of the determination in step S31 is 
affirmative, the process eventually proceeds to step S70 assuming that the 
determination of a copy-prohibited object is not difficult. If the result of the 
determination in step S31 is negative, the process proceeds to step S41. The 
process of outputting a signal requesting to change the mounting angle of the 
original in step S40 of FIG. 4 is replaced by the process of outputting a signal 
requesting to change the mounting position of the original as shown in step S41 
of FIG. 5. Other portions are entirely the same as in the first or second 
embodiment. 

[0060] Although in the above-described first through third embodiments, the 
calculation of difficulty in determination whether or not an image is a copy- 
prohibited object shown in step S20 or S21 is performed from an image obtained 
by pre-scanning shown in step S10, the present invention is not limited to such an 
approach. That is, actual scanning performed in step S50 may be performed in 
step S10. The image read in this actual scanning may be stored in the RAM 12 or 

13 



the external storage device 18, and the stored image may be processed at the 
processing starting from step S60. 



[0062] Although in the above-described first through fourth embodiments, the 
characteristics of a copy-prohibited object are checked for the image read by the 
scanner, the present invention is not limited to such an approach. 



[0066] Although in the above-described first through fifth embodiments, image 
data is output to the external storage device 18 (a hard disk or the like within the 
image processing system) via the I/O 19 as an image data file, whether the image 
data is output as normal image data or abnormal image data, the present 
invention is not limited to such an approach. 



[0070] In this case, output of image data in step S80 or S90 in the flowchart of 
FIG. 4 or 5 is realized by output of image data to the color printer 24 connected to 
the image processing system via the I/O 25, through connection cable 5. 



[0072] Although in the above-described first through sixth embodiments, the 
presence of characteristics indicating whether or not input image data represents 
a copy-prohibited object is determined, the present invention is not limited to 
such an approach. 
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[0073] That is, The present invention may also be applied to a case in which 
sub-information relating to image data itself included within the image data is 
detected, instead of a case of determining whether or not the image data 
represents a copy-prohibited object. For example, the present invention may also 
be applied to a case in which the presence of copyright information (sub- 
information) is confirmed from a hard-copy image for an image in which 
copyright information, serving as second digital information (sub-information), is 
mixed in original image data, serving as first digital information (main 
information), using the digital-watermark technique in order to claim the 
copyright of the user who has formed the image. By calculating the difficulty in 
extraction of the characteristics of the image as described above, and 
determining whether or not retry of input of the image is to be urged, based on 
the result of the calculation, it is, of course, possible to construct an image 
processing system which can easily extract the characteristics of the image. 
[0074] In this case, the copy-prohibited-object recognition unit 130 shown in 
FIG. 1 may be replaced by an image-characteristics extraction unit 131 shown in 
FIG. 9. Since other components shown in FIG. 9 operate in entirely the same 
manner as in the case of FIG. 1 , further detailed description thereof will be 
omitted. In accordance with this modification, as shown in FIG. 10, the 
processing in step S60 in FIG. 4 or 5 is replaced by processing of extracting image 
characteristics (sub-information) in step S61 of FIG. 10, and the processing in 
step S70 in FIG. 4 or 5 is replaced by determination processing in step S71 of FIG. 
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